Zero-copy network and file offload for web and application servers

ABSTRACT

Methods and apparatus for transferring data from an application server are provided. By offloading network and file system stacks to a common stack accessible by multiple operating systems in a virtual computing system, embodiments of the present invention may achieve data transfer support for web and application servers without the data needing to be copied to or reside in the address space of the server operating systems.

RELATED APPLICATIONS

The present invention claims benefit of provisional application Ser. No.60/693,133, entitled “Network Stack Offloading Approaches” filed on Jun.22, 2005, herein incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to computer networks and, moreparticularly, to offloading network and file services.

2. Description of the Related Art

The primary task of web and application servers is typically to retrievefiles from local file systems and send the file contents to remoteclients, for example, over established Transmission Control Protocol(TCP) connections. In a typical application, a remote client connectedto a server over a company Intranet or the Internet may request a filefrom a server. As an example, a web server may receive a request for afile (e.g., a document, image, HTML file, or the like) from a clientconnected via a TCP socket connection, retrieve the file, and send thefile to the requester.

FIG. 1 illustrates such a file access involving a conventional serversystem. An operating system (e.g., Linux, Windows, or some otheroperating system) usually segregates the available system memory (andother resources) into user space 110 and kernel space 120. The kernelspace is typically reserved for running the kernel, device drivers andany kernel extensions and includes file systems, network interfaceservices, and the like. On the other hand, user space includes memoryareas used by user applications. User applications cannot typicallyaccess the kernel space directly and, similarly, kernel code cannotaccess the user space without first checking whether the page is presentin memory or swapped out.

As illustrated, when a request for a file is received by an applicationserver 122, the application server 122 typically issues several readrequests to copy data from the file system buffer cache 130 in kernelspace 120 to buffers in user space 110. For example, in the kernel, therequested data may be first retrieved (e.g., from disk) and copied intothe file system buffer cache 130 and then copied into memory theapplication server 122 can access, such as a buffer created by theapplication server 122 for this purpose and specified with the readrequest.

After copying the data into user space, the application server 122 sendsthe data back out to buffers 140 (e.g., TCP socket buffers) in kernelspace 120. In this typical scenario, the application server 122 neverneeds to actually examine the data. Thus, copying the data from the filesystem buffer cache 130 to user space is inefficient, resulting inprocessing delays and consumption of resources (processing and memoryresources) in user space 110.

In an effort to more efficiently transfer data, mechanisms, such as theLinux sendfile command, have been created that attempt to avoidinefficient copying of data to and from user space. Rather than copyingread data from the file system buffer cache 130 into user space 110,only to send it back down, the application server 122 may issue a singlesendfile command specifying a descriptor of a file from which data is tobe read from (as well as a location within the file and number of bytesto read), and a descriptor of a socket on which the file will be sent.Because this transfer occurs within the kernel, a copy to and from userspace of the application server may be avoided.

Unfortunately, this approach still consumes kernel resources, requiringcopying data into the kernel file system buffer cache, writing out tosocket buffers, and is less than optimal. As an example, in virtualmachine systems running multiple operating system images, for securityreasons, care must be taken to ensure each operating system image cannotaccess memory allocated by the other operating system images. As aresult, copying data into kernel space (e.g., to process a sendfilecommand) typically entails mapping physical memory pages into theoperating system image virtual address space. This mapping is typicallya lengthy, processing intensive task involving access to operatingsystem image page tables.

Therefore, what is needed is a more efficient mechanism for transferringdata from a server, for example, that reduces the impact on local andkernel space resources.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide methods and apparatus fortransferring data from a server.

One embodiment provides a method for transferring data in response to acommand issued from an application server. The method generally includescreating an offload stack structure running on one or more processorcores that may or may not be separate from one or more processor coreson which an operating system image executing the application server isrunning, the offload stack structure providing at least network and filestack resources, providing an interface to share network and fileresources of the offload stack between multiple operating system imagesrunning on the multi-core system, and providing zero-copy data transfersupport for the application server by processing the command issued fromthe application server to the offload stack without the need to copydata targeted by the command to address space of the operating systemimage on which the application server is running.

Another embodiment provides a host system generally including aplurality of processing cores, one or more operating system imagesrunning on one or more of the processing cores, an application serverrunning on one of the operating system images, and an offload stackrunning on one or more processor cores separate from the one or moreprocessor cores on which the operating system image running theapplication server is running. The offload stack structure generallyprovides zero-copy data transfer support for the application server byprocessing commands issued from the application server without the needto copy data targeted by the command to address space of the operatingsystem image on which the application server is running.

Another embodiment provides a networked computing system generallyincluding one or more host systems, each comprising one or moreoperating system images and an application server running on at leastone of the operating system images, and a remote device coupled to theone or more host systems via a network connection and having an offloadstack. The offload stack generally provides zero-copy data transfersupport for the application server by processing commands issued fromthe application server without the need to copy data targeted by thecommand to address space of the operating system image on which theapplication server is running.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the presentinvention can be understood in detail, a more particular description ofthe invention, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1 illustrates an exemplary file access data path in conventionalnetwork systems in accordance with the prior art.

FIG. 2 illustrates a host utilizing an offload stack in accordance withone embodiment of the present invention.

FIG. 3 illustrates a detailed block diagram of an offload stack inaccordance with one embodiment of the present invention.

FIG. 4 is a flow diagram of exemplary operations for transferring datautilizing an offload stack in accordance with embodiments of the presentinvention.

FIGS. 5A and 5B illustrate exemplary file access data paths utilizing anoffload stack, in accordance with one embodiment of the presentinvention.

FIG. 6 illustrates an exemplary network system utilizing a remotelylocated offload stack, in accordance with one embodiment of the presentinvention.

DETAILED DESCRIPTION

By offloading network and file system stacks to a common stackaccessible by multiple operating system images in a virtual computingsystem, embodiments of the present invention may achieve data transfersupport for web and application servers without the data needing to becopied to or reside in the address space of the server operating systemimages. This “zero-copy” data transfer support may enable existingapplication servers to handle much larger volumes of traffic thanconventional techniques without requiring significant modification toexisting application server code.

To facilitate understanding, the following description will refer to avirtual computing environment in which multiple operating system imagesshare one or more processor cores, possibly on the same centralprocessing unit (CPU), as one type of application in which embodimentsof the present invention may be used to advantage. However, thoseskilled in the art will recognize that embodiments of the presentinvention may be more generally applied in a variety of computingenvironments to achieve efficient data transfer by distancing the datapath of requested data from the server receiving the request byoffloading network and file services.

An Exemplary Common Offload Stack

FIG. 2 illustrates an exemplary host system 200, in which a plurality ofoperating systems (or OS images) 210 share a common offload stack 220.The common offload stack may operate as described in previouslyreferenced provisional application Ser. No. 60/693,133, entitled“Network Stack Offloading Approaches” filed on Jun. 22, 2005. The hostsystem 200 may include a number of resources, such as multiple processorcores and memory, shared between the operating systems 210. For example,each operating system 210 may be allocated some portion of shared memoryand some portion or all processing bandwidth of one or more processorcores. Such a system is commonly referred to as a virtual systembecause, while the operating systems may share resources, they mayoperate independently, utilizing their allocated resources, as if eachwas operating in a separate computer system.

The growth of communications between devices (up to and beyond 1 Gbps)has led to rapidly increasing burdens on central processors (CPU), toimplement communication processing (TCP/IP) functions. Servers utilizingmultiple processors (or single CPUs with multiple processor cores) havebeen designed in an effort to achieve sufficient processor cycles toservice application requirements and management of the communicationsstack, whether TCP/IP, any other type of interfaces, includingperipheral stacks such as Fibre Channel, Infiniband, Firewire (IEEE1394), USB, Bluetooth, and wireless 802.11. Along with increases inbandwidth, security requirements for inspecting traffic flows andprotection are becoming more stringent and place a correspondinglyhigher burden on the CPU. Security issues are even more of a challengein platforms with multiple operating systems, when one operating systemseeks to modify space shared with another.

A variety of security functions rely on communications stacks, such asvirtual private network VPN (functions), antivirus, inline encryptionand message integrity, discovery, and identity protocols. Each of thesefunctions requires management, especially where they cross operatingsystem boundaries, resulting in a corresponding increase in CPU burden.For some embodiments of the present invention, however, the commonoffload stack 220 may offer a unified user interface that provides amechanism for operating systems to offload security and related networkmanagement functions. By utilizing virtualization, the common offloadstack 220 may provide a flexible open platform for network and I/Ocontrol functions acting, in some cases, as an integrated router betweenmultiple operating systems 210 of the host system 200. In some cases,the offload stack may even run on a remote system (as described infurther detail below, with reference to FIG. 6).

For some embodiments, the offload stack 220 may run on a separate coreof a multi-core processor or, for some cases, even on a separateprocessor or offload card (as will be described in greater detail belowwith reference to FIG. 5B). The offload stack 220 may control allnetwork and block storage interfaces, and can be shared between themultiple guest operating systems 210. In order to communicate with theoffload stack 220, each operating system 210 may include a common stackinterface front end (CSI F/E) 212, for example, implemented as a set ofkernel drivers. The CSI F/E 212 may allow applications running in theoperating systems 210 to communicate with network, block I/O, and fileinterfaces 221, 222, and 223, respectively, of the offload stack 220.

FIG. 3 illustrates the types of network, block I/O, and file functionsthat may be offloaded to the offload stack 220, and available to theoperating systems 210 via the CSI F/E 212, according to one embodimentof the present invention. As illustrated, the network offload functionsmay include any type of appropriate network processing functions, suchas RDMA 316, TCP 318, IP 320, and/or any other type of network functions322, such as Ethernet, Infiniband, and/or wireless protocols (e.g.,802.11). By providing access to the offload stack via a socketinterface, existing applications will not require changes to takeadvantage of offloading these functions.

As illustrated, the offload stack 220 may also provide file and blockI/O functions, via file interface 223 and block I/O interface 222,respectively. The file system types may include the common Internet filesystem (CIFS) 302, the network file system (NFS) 304, an object filesystem 306 (e.g., Lustre), cluster type file system 308 (e.g., GFS), anda local file system 310. The CSI FE may be configured such that the fileinterface 223 is accessible under a virtual file system VFS (virtualfile system) of the host 210.

The block I/O interface 222 may provide functional support for standardstorage interfaces. For example, a small computer system interface(SCSI) support 312 may allow connection of SCSI (or iSCSI) storagedevices, while a Fibre channel protocol (FCP) support 314 may allowconnection of a storage device via a Fibre Channel interface 324.

In essence, the offload stack 220 appears to the operating systems 210as being on the network. As a result, the network, file, and I/Ofunctionality allow the offload stack 220 to function, in effect, as anintermediate embedded network device capable of bridging, switching oreven routing between hosts on the server, and off the server whenoperating in conjunction with other (external) network devices deeper ina network.

Zero-Copy File Transfers Using an Offload Stack

As previously described, when transferring files requested fromapplication servers using conventional network communications, a highcost is incurred to copy data between kernel buffers and user processvirtual memory at the socket layer. However, by offloading network andfile system stacks (both network and local file system) in the samecommon offload stack, embodiments of the present invention may be ableto achieve “zero-copy” support for web and application servers byavoiding the need for data to be copied or even reside in the hostaddress space. As a result, servers utilizing this approach may be ableto handle significantly larger volumes of traffic than when utilizingconventional approaches and, in some cases, without significantmodification to server application code.

FIG. 4 is a flow diagram of exemplary operations 400 for performingzero-copy data transfer in accordance with embodiments of the presentinvention, utilizing an offload stack. The operations begin, at step402, by creating an offload stack running on a separate core of amulti-core machine. As described above, depending on the embodiment, theoffload stack may be running on a separate dedicated physical core,multiple physical cores, or a virtual core allocated some portion of oneor more physical cores.

At step 404, an interface to share the offload stack between multipleguest operating systems is provided. At step 406, zero-copy datatransfer support for an application server running on a guest operatingsystem is provided. Referring back to FIG. 2, the shared interface mayinclude a combined interface including access to offloaded networkservices (via interface 221), block I/O services (via interface 222),and file services (via 223). The guest operating systems may access theservices via a set of drivers, such as common shared interface (CSI)front end code also shown in FIG. 2. By offloading both the network andfile system stacks (both network and local file system) in the sameoffload engine, the zero-copy data transfer support may avoid the needfor targeted data to be copied to or reside in the guest operatingaddress space. As a result, an application server may be able to handlemuch larger volumes of traffic, without significant modification.

FIG. 5A illustrates how an offload stack 550 may allow a data path ofdata transferred by an application server 522 to be removed from theaddress space of the operating system running the application server. Inother words, the offload stack 550 may allow the application server 522to transfer data without copying the data to user space 510 or kernelspace 520, by sending a command (e.g., a sendfile command) specifying asource from which to read (e.g., a file or socket) and a writedestination (e.g., a socket). In response to the command, the offloadstack 550 in accordance with one embodiment of the present invention,may retrieve data from the specified source (e.g., into a file systembuffer cache 530) and send the data directly to the specified writedestination (e.g., to a socket connection via TCP socket buffers 540).

For some embodiments, a command issued by the application server 522 toachieve zero-copy support, may include a variety of arguments to specifywhat data to transfer and to where. For example, a sendfile commandissued by the application server 522 and compatible with the offloadstack may include arguments for specifying a descriptor of a file toread from (in_fd), a file to write to or stream socket on which the datawill be sent on (out_fd). The command may optionally include a pointerto an offset field which specifies an offset location in a specifiedfile to begin reading from (*offset), as well as a number of bytes toread/send (count). For some embodiments, to provide indication ofsuccessful completion of the command, the location of the byteimmediately following the last byte sent from the file may be written to*offset.

To enable complete offload of the sendfile command, the out_fd may beconverted to an offload stack specific file or socket identifier and thein_fd may be converted to an offload stack file identifier beforesending the sendfile command over to the offload stack. The offloadstack may then perform the complete I/O transaction on the offloadprocessor(s), whether real or virtual. When the transaction hascompleted, status may be returned to the guest domain (including thevalue to be written to *offset) so that the originating sendfile( ) callcan return appropriate status to the calling user application.

As illustrated in FIG. 5B, for some embodiments, the transfer data pathmay be moved even farther from the application server issuing thesendfile command, by offloading the offload stack 550 to a separate datapath processor 560. Depending on the particular embodiment, the datapath processor 560 may be a completely separate processor and may evenbe running on a separate “offload engine” interface card.

For some embodiments, the transfer data path may be even further removedfrom application servers issuing sendfile commands. For example, asillustrated in FIG. 6, a “remote” offload stack 650 may be offloaded toa separate processor on a remote device 680 with a network connection tohosts 600 running the application servers. As illustrated, for someembodiments, “local” offload stacks 620 may handle local devices andfunctions, passing shared network, block, and file service requestsdirectly to the offload stack 650 located on the remote device 680.

In the illustrated arrangement, network and file services may beoffloaded to the remote offload stack 650 on a network device, such as agateway. In such cases, network, block and I/O interfaces may be passeddirectly over whatever type of medium is used for the network connection692, such as Infiniband or a DataCenter Ethernet, and the gateway device680 may include a compatible switch 690. Such an offload stack 650 on anetwork gateway may handle shared network, file, and I/O services for anentire cluster of server hosts 600. As a result, an application serveron a host 600 may be able to transfer data via a zero-copy sendfilecommand, even if the data only exists in the remote offload stack 650.

CONCLUSION

Embodiments of the present invention may achieve data transfer supportfor web and application servers without the data needing to be copied toor reside in the address space of the server operating systems. This“zero-copy” data transfer support may be achieved by offloading networkand file system stacks to a common stack accessible by multipleoperating systems in a virtual computing system and may enable anapplication server to handle larger volumes of traffic than achievableusing conventional techniques requiring data to be copied into addressspace of the operating system running the application server.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the invention may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

1. A method for transferring data in response to a command issued froman application server, comprising: creating an offload stack structurerunning on one or more processor cores separate from one or moreprocessor cores on which an operating system executing the applicationserver is running, the offload stack structure providing at leastnetwork and file stack resources; providing an interface to sharenetwork and file resources of the offload stack between multipleoperating systems running on the multi-core system; and providingzero-copy data transfer support for the application server by processingthe command issued from the application server to the offload stackwithout the need to copy data targeted by the command to address spaceof the operating system on which the application server is running. 2.The method of claim 1, wherein the offload stack provides network, fileand block I/O resources.
 3. The method of claim 1, wherein the commandincludes at least a source argument to specify a source of data to betransferred and a destination argument to which the data is to betransferred.
 4. The method of claim 3, wherein the command also includesat least one argument specifying an offset location within a filespecified by the source argument from which to start reading.
 5. Themethod of claim 3, wherein the source argument is capable of specifyinga file or a socket stream.
 6. The method of claim 1, wherein theinterface to share network and file resources of the offload stackbetween multiple operating systems running on the multi-core systemcomprises driver code executable by the operating systems.
 7. The methodof claim 1, wherein creating the offload stack structure comprisescreating the offload stack structure on a data path processor separatefrom any processor on which any of the operating systems are running. 8.The method of claim 7, wherein creating the offload stack structurecomprises creating the offload stack structure on a network devicecoupled to the host device via a network connection.
 9. A host system,comprising: one or more processing cores; one or more operating systemsrunning on one or more of the processing cores; an application serverrunning on one of the operating systems; and an offload stack running onone or more of the processor cores, the offload stack structureproviding zero-copy data transfer support for the application server byprocessing a command issued from the application server without the needto copy data targeted by the command to address space of the operatingsystem on which the application server is running.
 10. The host systemof claim 9, wherein the application and offload stack are running ondifferent processor cores.
 11. The host system of claim 9, wherein theoffload stack provides network, file and block I/O resources to theoperating systems.
 12. The host system of claim 9, wherein the commandincludes at least a source argument to specify a source of data to betransferred and a destination argument to which the data is to betransferred.
 13. The host system of claim 12, wherein the command alsoincludes at least one argument specifying an offset location within afile specified by the source argument from which to start reading. 14.The host system of claim 13, wherein the source argument is capable ofspecifying a file or a socket stream.
 15. The host system of claim 9,wherein the offload stack structure runs on a data path processorseparate from any processor on which any of the operating systems arerunning.
 16. A device, comprising: an interface allowing the device tobe coupled to one or more host systems via a network connection; and anoffload stack providing zero-copy data transfer support for anapplication server running on an operating system of at least one of thehost systems by processing a command issued from the application serverwithout the need to copy data targeted by the command to address spaceof the operating system on which the application server is running. 17.The device of claim 16, wherein the network connection comprises atleast one of: a Data Center Ethernet (DCE) connection and an Infinibandconnection.
 18. The device of claim 16, wherein the offload stackprovides network, file and block I/O resources to the host systems. 19.The device of claim 16, wherein the offload stack provides theapplication server with zero-copy support to transfer data from a remotefile system to a socket connection with a remote client.
 20. The deviceof claim 16, wherein the command includes at least a source argument tospecify a source of data to be transferred and a destination argument towhich the data is to be transferred.
 21. The device of claim 20, whereinthe source argument is capable of specifying a file or a socket stream.22. A system, comprising: one or more processing means; server meansrunning on at least one of the one or more processing means; and offloadmeans running on at least one of the one or more processing means, theoffload means providing zero-copy data transfer support for the servermeans by processing commands issued from the server means without theneed to copy data targeted by the commands to address space of anoperating system on which the server means is running.
 23. The system ofclaim 22, wherein the offload means provides network, file and block I/Oresources to the operating systems.
 24. The system of claim 22, whereinthe offload means runs on processor means separate from any processormeans on which the server runs.